Cloud edition and retrieve

ABSTRACT

A cloud instantiation of a secondary storage system is generated. An indication to virtually rebuild one or more secondary storage clusters based on a snapshot archive stored in cloud storage is received. The one or more secondary storage clusters are virtually rebuilt in the cloud instantiation of the secondary storage system, including by reconstituting a tree data structure based on the snapshot archive. A copy of at least a portion of data of the one or more rebuilt storage clusters in the cloud instantiation to restore at least the portion of data is provided to a user destination system.

BACKGROUND OF THE INVENTION

A primary system may be configured to perform a backup snapshot of itsfile system data to a secondary storage system. The secondary storagesystem may store the backup snapshot. The primary system may request torestore from the secondary storage system any of the file system dataincluded in the backup snapshot. In some instances, the secondarystorage system may go offline. As a result, the primary system is unableto restore from the secondary storage system any of the file system dataincluded in the backup snapshot and must wait until the secondarystorage comes back online.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a distributedstorage system.

FIG. 2A is a block diagram illustrating an embodiment of a tree datastructure storing file system data.

FIG. 2B is a block diagram illustrating an embodiment of cloning asnapshot tree of a tree data structure.

FIG. 2C is a block diagram illustrating an embodiment of modifying asnapshot tree.

FIG. 2D is a block diagram illustrating an embodiment of a modifiedsnapshot tree of a tree data structure.

FIG. 2E is a block diagram illustrating an embodiment of file systemdata.

FIG. 3A is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time.

FIG. 3B is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time.

FIG. 3C is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time.

FIG. 3D is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time.

FIG. 4A is a block diagram illustrating an embodiment of archive data.

FIG. 4B is a block diagram illustrating an embodiment of archive data.

FIG. 4C is a block diagram illustrating an embodiment of archive data.

FIG. 4D is a block diagram illustrating an embodiment of archive data.

FIG. 5 is a flow chart illustrating an embodiment of archiving data.

FIG. 6 is a flow chart illustrating an embodiment of a process forrestoring data.

FIG. 7 is a flow chart illustrating an embodiment of restoring archiveddata.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

A primary system comprises an object, virtual machine, physical entity,file system, array backup, and/or volume that stores file system data.The primary system may perform a backup snapshot according to a backuppolicy and store the backup snapshot to a secondary storage system. Abackup snapshot represents the state of the primary system at aparticular point in time (e.g., the state of the file system data). Thebackup policy may require a full backup snapshot and/or an incrementalbackup snapshot to be performed. A full backup snapshot stores theentire state of the primary system at a particular point in time. Thefile system data is comprised of a plurality of data blocks. The primarysystem may include a change block tracker that monitors and identifiesone or more data blocks of the primary system that have changed (e.g.,modified, deleted, added) since the last backup snapshot. An incrementalbackup snapshot includes the one or more data blocks that have changedsince the last backup snapshot.

The one or more data blocks of a backup snapshot that are stored on asecondary storage system may be organized using a tree data structure.An example of the tree data structure is a snapshot tree (e.g., CohesitySnaptree), which may be based on a B+ tree structure (or other type oftree structure in other embodiments). The snapshot tree enables a backupsnapshot to be a fully hydrated backup snapshot, i.e., a backup snapshotthat provides a complete view of the primary system at a moment in timecorresponding to when the backup snapshot was performed. The snapshottree may be used to capture different versions of file system data atdifferent moments in time. For example, a first snapshot tree maycorrespond to a first backup snapshot and a second snapshot tree maycorrespond to a second backup snapshot. In some embodiments, the treedata structure allows a chain of snapshot trees (i.e., differentsnapshot tree versions) to be linked together by allowing a node of alater version of a snapshot tree to reference a node of a previousversion of a snapshot tree (e.g., a “snapshot tree forest”). Forexample, a node of the second snapshot tree corresponding to the secondbackup snapshot may reference a node of the first snapshot treecorresponding to the first backup snapshot.

A snapshot tree includes a root node, one or more levels of one or moreintermediate nodes associated with the root node, and one or more leafnodes associated with an intermediate node of the lowest intermediatelevel. A root node or an intermediate node of a version of a snapshottree may reference an intermediate node or a leaf node of a previousversion of a snapshot tree. The root node of a snapshot tree includesone or more pointers to one or more intermediate nodes. Eachintermediate node includes one or more pointers to other nodes (e.g., alower intermediate node or a leaf node). Each time a backup snapshot isreceived by the secondary storage system, a corresponding snapshot treeis created. The corresponding snapshot tree is created by cloning thelast snapshot tree. Cloning the last snapshot tree includes creating anew root node where the new root node includes the set of pointersincluded in the root node of the last snapshot tree, that is, the newroot node includes one or more pointers to an intermediate node or leafnode associated with the last snapshot tree. The new root node alsoincludes a view identifier (e.g., TreeID) that is different from theview identifier of the last root node. Each node of a snapshot tree hasa corresponding view identifier. The view identifier associated with anode identifies a view with which the node is associated, e.g., theassociated backup snapshot. A leaf node may store file system metadata,one or more data blocks, an identifier of one or more data blocks, apointer to a file tree, or a pointer to a data block stored on thesecondary storage system.

It is desirable to periodically archive the tree data structure toarchive changes and provide fault tolerance. A backup snapshot may bearchived from the secondary storage system to a cloud (e.g., Amazon WebServices, Microsoft Azure, Google Cloud, etc.). A backup snapshot may bearchived to the cloud based on one or more archive policies. An archivepolicy may indicate that a backup snapshot is to be archived after beingstored on the secondary storage system for a threshold duration (e.g.,30 days). The archive policy may indicate that a full snapshot archiveof the file system data and/or an incremental snapshot archive of thefile system data is to be performed. A full snapshot archive is anarchive of a full backup snapshot. It includes all the nodes of asnapshot tree without referring back to the node of a previous snapshottree. An incremental snapshot archive is an archive of an incrementalbackup snapshot. It includes the nodes that represent the differencebetween a backup snapshot and a previous backup snapshot and one or morereferences to one or more nodes associated with one or more previoussnapshots.

If the storage where the tree data structure is to be archived nativelyunderstands and supports tree data structures, the backup snapshot canbe directly copied to the archive storage in a native format. However,many storage solutions do not natively support tree data structures. Thestorage solutions that natively support tree data structures are oftencostly and inefficient for use as an archive.

It may be more efficient to archive data to a storage/archivemedium/system that is unable to natively support the tree datastructure. For example, traditional third-party cloud solutions providethe ability to store computer files in a reliable manner but lack theability to natively understand and support a tree data structure (e.g.,do not natively provide the ability to perform tree data structuretransactions). A snapshot archive may include file system data andserialized snapshot tree data. File system data includes one or moredata chunks that make up data of a file system. In some embodiments, aversion of file system data is comprised of one or more blocks that aresegmented into one or more chunks of data, a chunk identifier iscomputed for each chunk of data, and the chunk identifiers are stored asfile system metadata.

In addition to archiving the file system data, a correspondingserialized snapshot tree data is archived to the cloud. Serializing is aprocess by which a data file is generated to store data in a manner thatmimics the structure of a tree data structure. The serialized data filemay be encoded in a manner that allows the serialized data file to beutilized to reconstruct a desired portion of the tree data structure toobtain a data of interest from the serialized data file without the needto reconstruct the entire tree data structure. A serialized snapshottree data stores the structure of the snapshot tree associated with thefile system data as a flat set of data that is comprised of one or moreblocks. Each block of the flat set of data corresponds to a node of thesnapshot tree. The order of the data blocks of the serialized data filecorresponds to an order of the tree data structure. A tree datastructure may have a root node, a plurality of intermediate nodes, and aplurality of leaf nodes. The serialized data file first may includeblocks corresponding to the root node, then the plurality ofintermediate nodes, and then the plurality of leaf nodes. For example, afirst data block of the serialized data file may correspond to a rootnode of the tree data structure, a second data block of the serializeddata file may correspond to a first intermediate node of the tree datastructure, a third data block of the serialized data file may correspondto a second intermediate node of the tree data structure, a fourth datablock of the serialized data file may correspond to a first leaf node ofthe tree data structure, . . . and an nth data block of the serializeddata file may correspond to the nth leaf node of the tree datastructure. The blocks that correspond to an intermediate node or a rootnode may include a file offset to another block of the serializedsnapshot tree data or a file offset to another block of a differentserialized snapshot tree data. A block that corresponds to a leaf nodemay include a file offset to a portion of the archived file system data.

A user of the primary system may request to restore from cloud storage asnapshot archive corresponding to a particular state of the primarysystem. Normally, a secondary storage system may receive the snapshotarchive from cloud storage and reconstitute a snapshot tree associatedwith the snapshot archive by deserializing the serialized data file. Theuser of the primary system may request file system data associated withthe reconstituted snapshot tree. The reconstituted snapshot tree may betraversed and the requested file system data is provided to the primarysystem.

However, in some instances, the secondary storage system may be offlineand the secondary storage system is unable to restore the snapshotarchive from the cloud. For example, a physical component of thesecondary storage system may have failed and needs to be replaced. Itmay take a particular period of time before the physical component isreplaced (e.g., due to shipping time and/or repair time). The primarysystem may be unable to deserialize the snapshot archive from the cloud,that is, the snapshot archive is in a format that the primary system isunable to understand. As a result, a user of the primary system may haveto wait for a period (e.g., days, weeks) before the secondary storagesystem comes back online. This causes the primary system to accrueundesirable downtime and may delay the primary system from performingone or more tasks associated with the file system data included in thesnapshot archive.

A cloud edition, i.e., a cloud instantiation, of the secondary storagesystem may be generated. A configuration of cloud instantiation of thesecondary storage system may be a replica of the secondary storagesystem. For example, an on-prem secondary storage system may becomprised of three nodes, each node with a storage capacity of 10 TB. Acloud instantiation of the secondary storage system may be comprised ofthree virtual nodes, each virtual node with a storage capacity of 10 TB.The user associated with the primary system may be authenticated. Theuser may be provided a list of one or more snapshot archives to whichthe user has access. A request to restore one or more snapshot archivesmay be received at a cloud portal associated with the secondary storagesystem from the user associated with a primary system. The request maybe for a full snapshot archive or an incremental snapshot archive.

The one or more requested snapshot archives may be received at the cloudinstantiation of the secondary storage system. The cloud instantiationof the secondary storage system may include a virtual file systemmanager that is configured to virtually rebuild the secondary storagesystem at least in part by reconstituting the requested snapshotarchive. In some embodiments, a snapshot tree associated with thesnapshot archive is reconstituted by deserializing the serializedsnapshot tree data included in the received snapshot archive. Forexample, the received snapshot archive corresponds to a full snapshotarchive. In other embodiments, a snapshot tree associated with thesnapshot archive is reconstituted by deserializing serialized snapshottree data included in the received snapshot archive and deserializingone or more other serialized snapshot tree data included in one or moreother received snapshot archives. For example, the received snapshotarchive corresponds to an incremental snapshot archive and the one ormore other received snapshot archives may correspond to full orincremental snapshot archives to which the incremental snapshot archiverefers.

Reconstituting the structure of a snapshot tree associated with thesnapshot archive for the particular time includes reading the flat setof data associated with the serialized snapshot tree data included inthe received snapshot archive. The flat set of data includes blocks ofdata that correspond to nodes of a snapshot tree and associated fileoffsets that correspond to pointers of the snapshot tree. The reproducedtree data structure is traversed. The file system data associated withthe requested snapshot archive is provided to the primary system. Inresponse to receiving the file system data, the primary system is ableto restore itself back to a particular moment in time, i.e., aparticular state.

The cloud instantiation of the secondary storage system reduces theamount of downtime of the primary system when the secondary storagesystem is offline because the primary system does not have to wait forthe secondary storage system to come back online. It may take days, evenweeks, before the secondary storage system is back online. The cloudinstantiation of the secondary storage system may be generated withinminutes and may enable the primary system to be restored to a particularstate within minutes and/or hours.

FIG. 1 is a block diagram illustrating an embodiment of a distributedstorage system. In the example shown, system 100 includes data center101 coupled to cloud 121 via network 111.

Data center 101 is comprised of primary system 102 and a secondarystorage system 104. Primary system 102 is a computing system that storesfile system data. Primary system 102 may be comprised of one or moreservers, one or more computing devices, one or more storage devices,and/or a combination thereof. Primary system 102 may be configured tosend a backup snapshot of file system data to secondary storage system104 according to one or more backup snapshot policies. In someembodiments, a backup snapshot policy indicates that file system data isto be backed up on a periodic basis (e.g., hourly, daily, weekly,monthly, etc.). In other embodiments, a backup snapshot policy indicatesthat file system data is to be backed up when a threshold size of datahas changed. In other embodiments, a backup snapshot policy indicatesthat file system data is to be backed up upon a command from a userassociated with primary system 102.

Secondary storage system 104 is a storage system configured to storefile system data included in a backup snapshot received from primarysystem 102. Secondary storage system 104 may also be referred to as asecondary storage cluster. Secondary storage system 104 may beconfigured to implement policy-based archival of older backup snapshotsfrom on-prem to cloud 121 for long-term retention. The cloud archiveallows data to be indexed for fast search and retrieval back to on-premfrom the cloud. The cloud archive allows recovery of data to a differentsite (e.g., another computing system) in the event primary system 102and/or secondary storage system 104 fails. The cloud archive may allowdata encryption in-flight and at-rest in the cloud. Secondary storagesystem 104 may be configured to archive a copy of a backup snapshot tocloud 121 for disaster recovery. Secondary storage system 104 may beconfigured to move certain file system data to cloud 121 to handlespikes in storage demand. Secondary storage system 104 may be configuredto implement a policy-based waterfall model that moves cold data tocloud 121. Upon receiving a read for data that is stored at cloud 121,secondary storage system 104 is configured to retrieve the data andstore the data at the secondary storage system 104.

Secondary storage system 104 is configured to organize the file systemdata in a tree data structure and to create a snapshot tree based on thefile system data. The snapshot tree may provide a view of primary system102 at a particular moment in time. A view depicts the connectionsbetween nodes and the data stored in one or more leaf nodes at theparticular moment in time. The tree data structure allows a chain ofsnapshot trees to be linked together. Each time a backup snapshot isperformed, a root node of the snapshot tree may be linked to one or moreintermediate nodes associated with a previous snapshot tree. Secondarystorage system 104 may archive the file system data to cloud 121. Thearchive may include the file system data and a serialized snapshot treedata that is a serialized version of the tree data structure. In someembodiments, the archive includes a full snapshot archive of the filesystem data. In other embodiments, the archive includes an incrementalsnapshot archive of the file system data.

Secondary storage system 104 may be configured to archive any of thedata stored on secondary storage system 104 (e.g., tree data, othernon-tree data) according to one or more archive policies. In someembodiments, an archive policy indicates that the data is to be archivedto cloud 121 on a periodic basis (e.g., hourly, daily, weekly, monthly,etc.). In other embodiments, an archive policy indicates that data is tobe archived to cloud 121 when a threshold size of data has changed. Inother embodiments, an archive policy indicates that data is to bearchived to cloud 121 upon a command from a user associated withsecondary storage system 104.

Secondary storage system 104 may be comprised of a plurality of storagenodes. The plurality of storage nodes may comprise one or more solidstate drives, one or more hard disk drives, or a combination thereof.Secondary storage system 104 may include one or more processors coupledto the storage drives and configured to archive data stored in thestorage drives to cloud 121.

Secondary storage system 104 may include file system manager 105. Filesystem manager 105 is configured to maintain file system data in theform of nodes arranged in a tree data structure. In some embodiments,leaf nodes in the snapshot tree include key-value pairs that associatedata keys with values in the form of particular items of file systemdata. In other embodiments leaf nodes in the snapshot tree include areference to a file tree that stores the metadata associated with afile. A user may request a particular item of data by providing a datakey to file system manager 105, which traverses a snapshot tree to findthe item of data associated with that data key. File system manager 105may be configured to generate a snapshot archive of a snapshot tree. Thesnapshot archive includes a serialized version of the snapshot tree.File system manager 105 may be configured to perform one or moremodifications, as disclosed herein, to a snapshot tree.

Data center 101 is coupled to cloud 121 via network 111. Network 111 maybe one or more of the following: a local area network, a wide areanetwork, a wired network, a wireless network, the Internet, an intranet,or any other appropriate communication network.

Cloud 121 may be a public cloud provider (e.g., Amazon Web Services,Microsoft Azure, Google Cloud). Cloud 121 is configured to receive asnapshot archive from secondary storage system 104 and store thesnapshot archive in cloud object storage 124. Cloud object storage 124may store a full snapshot of file system data and associated serializedsnapshot tree data. Cloud object storage 124 may store an incrementalsnapshot of file system data and associated serialized snapshot treedata.

Cloud 121 may include cloud portal 123. Cloud portal 123 is associatedwith one or more secondary storage systems. Cloud portal 123 may providea user associated with access to any snapshot archive to which the useris permitted.

An enterprise may be associated with one or more data centers. Each datacenter may have a corresponding secondary storage system. Thecorresponding secondary storage systems may be configured to archivecorresponding backup snapshots to cloud 121 based on one or more archivepolicies. A user associated with the enterprise may be permitted toaccess a snapshot archive and request a snapshot archive to be restoredto one of the one or more data centers associated with the enterprise.In other embodiments, the user is associated with one of theenterprise's data centers. The user may be permitted to access snapshotarchives specific to that particular data center and restore to aprimary system of the particular data center or the secondary storagesystem of the particular data center, the snapshot archives specific tothat particular data center.

Cloud portal 123 is configured to authenticate a user associated with anenterprise. Cloud portal 123 may request the user associated with theenterprise to provide a credential that indicates the user is permittedto access the one or more snapshot archives associated with theenterprise.

Cloud portal 123 may cause a cloud instantiation 122 of secondarystorage system 104 to be generated in response to a user request.Secondary storage system 104 is comprised of a plurality of storagenodes. Each storage node has a particular storage capacity. Cloud portal123 may be configured to cause cloud instantiation 122 of secondarystorage system 104 to have the same storage capacity as secondarystorage system 104. For example, secondary storage system 104 may becomprised of three physical storage nodes, each physical storage havinga storage capacity of 10 TB. Cloud portal 123 may be configured togenerate cloud instantiation 122 to include three virtual cloudinstances, each virtual cloud instance (e.g., a virtual machine) havinga storage capacity of 10 TB. Cloud instantiation 122 may be configuredfor the public cloud (e.g., Amazon Web Services, Microsoft Azure, GoogleCloud) in which cloud instantiation 122 will reside. In otherembodiments, cloud instantiation 122 of secondary storage system 104 mayhave more storage capacity than secondary storage system 104. In otherembodiments, cloud instantiation 122 of secondary storage system 104 mayhave less storage capacity than secondary storage system 104.

In response to validating the user's credential, a cloud user interfacemay provide to the user a list of one or more snapshot archivesassociated with the enterprise. The cloud user interface may receive aselection of a snapshot archive and an associated destination for thefile system data associated with the selected snapshot archive.

Cloud object storage 124 is configured to store a plurality of snapshotarchives. A subset of the snapshot archives may be associated with theenterprise associated with data center 101. For example, cloud objectstorage 124 is configured to store snapshot archives associated with aplurality of enterprises. Cloud object storage 124 may receive a requestfor one of the stored snapshot archives. In response to the request,cloud object storage 124 is configured to provide the requested snapshotarchive to the cloud instantiation associated with the request, forexample, cloud instantiation 122. Cloud 121 may include a plurality ofcloud instantiations of secondary storage systems. Each cloudinstantiation may correspond to a particular data center. An enterprisemay be associated with a plurality of cloud instantiations. A pluralityof enterprises may be associated with corresponding cloudinstantiations.

Cloud instantiation 122 may include virtual file system manager 125.Cloud instantiation 122 may receive a snapshot archive from cloud objectstorage 124. Virtual file system manager 125 is configured todeserialize the serialized snapshot tree data included in the snapshotarchive and reconstitute a snapshot tree associated with the snapshotarchive. In some embodiments, virtual file system manager 125 isconfigured to reconstitute an entire snapshot tree because theassociated snapshot archive is a full snapshot archive. In otherembodiments, virtual file system manager 125 is configured toreconstitute a partial snapshot tree because the associated snapshotarchive is an incremental snapshot archive. In other embodiments, aplurality of snapshot archives are received (e.g., incremental snapshotarchives, one full snapshot archive) and virtual file system manager 125is configured to reconstitute the entire snapshot tree based on theplurality of snapshot archives. The deserialized snapshot tree data maybe stored across the virtual cloud instances of cloud instantiation 122.

Cloud storage 122 is configured to provide the file system dataassociated with the deserialized snapshot tree data to a requesteddestination. In some embodiments, the deserialized snapshot tree data isprovided to a primary system, such as primary system 102. In otherembodiments, the deserialized snapshot tree data is provided to asecondary storage system (e.g., a new secondary storage system isbrought online).

FIG. 2A is a block diagram illustrating an embodiment of a tree datastructure storing file system data. In the example shown, tree datastructure 200 may be created by a storage system, such as secondarystorage system 104. The file system data of a primary system, such asprimary system 102, may be backed up to a secondary storage system, suchas secondary storage system 104. The primary system may perform a backupsnapshot to back up the file system data. The backup snapshot maycorrespond to a full backup snapshot or an incremental backup snapshot.The manner in which the file system data is stored in a secondarystorage system may be represented by a corresponding tree datastructure, such as tree data structure 200.

In the example shown, tree data structure 200 is a snapshot tree thatincludes a root node 202, intermediate nodes 212, 214, and leaf nodes222, 224, 226, 228, and 230. Although tree data structure 200 includesone intermediate level between root node 202 and leaf nodes 222, 224,226, 228, and 230, there could be any number of intermediate levels in asnapshot tree. Tree data structure 200 may be a snapshot tree of filesystem data at a particular point in time t and correspond to a versionof a snapshot tree.

Root node 202 is associated with the backup snapshot of the file systemdata at the particular point in time t, i.e., a particular backupsnapshot version. In some embodiments, the file system data is metadatafor a distributed file system and may include information such as filesize, directory structure, file permissions, physical storage locationsof the files, etc.

A root node is the root of a snapshot tree and may include pointers toone or more other nodes. Each pointer includes a range of numerical andsorted data keys that can be found by following that particular pointer.An intermediate node is a node to which another node points (e.g., rootnode, other intermediate node) and includes one or more pointers to oneor more other nodes. A leaf node is a node at the bottom of a snapshottree. In some embodiments, a leaf node is a node to which another nodepoints, but does not include a pointer to any other node. In otherembodiments, a leaf node is a node to which another node points andincludes a pointer to the root node of a file metadata tree (e.g., asnapshot tree for a particular file). A leaf node may store key-valuepairs of file system data. A data key k is a lookup value by which aparticular leaf node may be accessed. A file may be comprised of one ormore data blocks. The file may be divided into a plurality of equal sizeblocks called bricks. The lookup value may correspond to a brick number.For example, “1” is a data key that may be used to lookup “DATA1” ofleaf node 222. Examples of values stored by a leaf node include, but arenot limited to, file size, directory structure, file permissions,physical storage locations of the files, etc. A leaf node may store adata key k and a pointer to a location that stores the value associatedwith the data key. The value may correspond to a data brick comprisingone or more data blocks.

A root node or an intermediate node may include one or more node keys.The node key may be an integer value or a non-integer value. The nodekey may correspond to a brick number. Each node key indicates a divisionbetween the branches of the node and indicates how to traverse the treedata structure to find a leaf node, i.e., which pointer to follow. Forexample, root node 202 may include a node key of “3.” A data key k of akey-value pair that is less than or equal to the node key is associatedwith a first branch of the node and a data key k of a key-value pairthat is greater than the node key is associated with a second branch ofthe node. In the above example, to find a leaf node storing a valueassociated with a data key of “1,” “2,” or “3,” the first branch of rootnode 202 would be traversed to intermediate node 212 because the datakeys of“1,” “2,” and “3” are less than or equal to the node key “3.” Tofind a leaf node storing a value associated with a data key of “4” or“5,” the second branch of root node 202 would be traversed tointermediate node 214 because data keys “4” and “5” are greater than thenode key of “3.”

In some embodiments, a hash function may determine which branch of anode with which the non-numerical data key is associated. For example, ahash function may determine that a first bucket is associated with afirst branch of a node and a second bucket is associated with a secondbranch of the node.

A data key k of a key-value pair is not limited to a numerical value. Insome embodiments, non-numerical data keys may be used for a datakey-value pair (e.g., “name,” “age,” etc.) and a numerical number may beassociated with the non-numerical data key. For example, a data key of“name” may correspond to a numerical key of“4.” Data keys thatalphabetically come before the word “name” or is the word “name” may befound following a left branch associated with a node. Data keys thatalphabetically come after the word “name” may be found by following aright branch associated with the node. In some embodiments, a hashfunction may be associated with the non-numerical data key. The hashfunction may determine which branch of a node with which thenon-numerical data key is associated.

In the example shown, root node 202 includes a pointer to intermediatenode 212 and a pointer to intermediate node 214. Root node 202 includesa NodeID of “R1” and a TreeID of “1.” The NodeID identifies the name ofthe node. The TreeID identifies the snapshot/view with which the node isassociated. When a change is made to data stored in a leaf node asdescribed with respect to FIGS. 2B, 2C, and 2D, the TreeID is used todetermine whether a copy of a node is to be made.

Root node 202 includes a node key that divides a set of pointers intotwo different subsets. Leaf nodes (e.g., “1-3”) with a data key k thatis less than or equal to the node key are associated with a first branchand leaf nodes (e.g., “4-5”) with a data key k that is greater than thenode key are associated with a second branch. Leaf nodes with a data keyof “1,” “2,” or “3” may be found by traversing tree data structure 200from root node 202 to intermediate node 212 because the data keys have avalue that is less than or equal to the node key. Leaf nodes with a datakey of “4” or “5” may be found by traversing tree data structure 200from root node 202 to intermediate node 214 because the data keys have avalue that is greater than the node key.

Root node 202 includes a first set of pointers. The first set ofpointers associated with a data key less than or equal to the node key(e.g., “1,” “2,” or “3”) indicates that traversing tree data structure200 from root node 202 to intermediate node 212 will lead to a leaf nodewith a data key of “1,” “2,” or “3.” Intermediate node 214 includes asecond set of pointers. The second set of pointers associated with adata key greater than the node key indicates that traversing tree datastructure 200 from root node 202 to intermediate node 214 will lead to aleaf node with a data key of “4” or “5.”

Intermediate node 212 includes a pointer to leaf node 222, a pointer toleaf node 224, and a pointer to leaf node 226. Intermediate node 212includes a NodeID of “I1” and a TreeID of “1.” Intermediate node 212includes a first node key of “1” and a second node key of “2.” The datakey k for leaf node 222 is a value that is less than or equal to thefirst node key. The data key k for leaf node 224 is a value that isgreater than the first node key and less than or equal to the secondnode key. The data key k for leaf node 226 is a value that is greaterthan the second node key. The pointer to leaf node 222 indicates thattraversing tree data structure 200 from intermediate node 212 to leafnode 222 will lead to the node with a data key of “1.” The pointer toleaf node 224 indicates that traversing tree data structure 200 fromintermediate node 212 to leaf node 224 will lead to the node with a datakey of “2.” The pointer to leaf node 226 indicates that traversing treedata structure 200 from intermediate node 212 to leaf node 226 will leadto the node with a data key of “3.”

Intermediate node 214 includes a pointer to leaf node 228 and a pointerto leaf node 230. Intermediate node 214 includes a NodeID of “I2” and aTreeID of “1.” Intermediate node 214 includes a node key of “4.” Thedata key k for leaf node 228 is a value that is less than or equal tothe node key. The data key k for leaf node 230 is a value that isgreater than the node key. The pointer to leaf node 228 indicates thattraversing tree data structure 200 from intermediate node 214 to leafnode 228 will lead to the node with a data key of “4.” The pointer toleaf node 230 indicates that traversing tree data structure 200 fromintermediate node 214 to leaf node 230 will lead to the node with a datakey of “5.”

A file may be comprised of a plurality of data blocks. The file may bedivided into a plurality of fixed size data blocks called bricks.Information associated with a brick for a file may be kept in the treedata structure. For example, a leaf node may correspond to a brick andinclude a pointer to a location of the corresponding one or more datablocks stored in the brick. Given an offset in a file, the brick numbermay be determined by dividing the file offset by a brick size. The bricknumber may correspond to a data key k for a leaf node.

Leaf node 222 includes a data key-value pair of “1: DATA1.” Leaf node222 includes NodeID of “L1” and a TreeID of “1.” To view the valueassociated with a data key of “1,” tree data structure 200 is traversedfrom root node 202 to intermediate node 212 to leaf node 222.

Leaf node 224 includes a data key-value pair of “2: DATA2.” Leaf node224 includes NodeID of “L2” and a TreeID of “1.” To view the valueassociated with a data key of “2,” tree data structure 200 is traversedfrom root node 202 to intermediate node 212 to leaf node 224.

Leaf node 226 includes a data key-value pair of “3: DATA3.” Leaf node226 includes NodeID of “L3” and a TreeID of “1.” To view the valueassociated with a data key of “3,” tree data structure 200 is traversedfrom root node 202 to intermediate node 212 to leaf node 226.

Leaf node 228 includes a data key-value pair of “4: DATA4.” Leaf node228 includes NodeID of “L4” and a TreeID of “1.” To view the valueassociated with a data key of “4,” tree data structure 200 is traversedfrom root node 202 to intermediate node 214 to leaf node 228.

Leaf node 230 includes a data key-value pair of “5: DATA5.” Leaf node230 includes NodeID of “L5” and a TreeID of “1.” To view the valueassociated with a data key of “5,” tree data structure 200 is traversedfrom root node 202 to intermediate node 214 to leaf node 230.

FIG. 2B is a block diagram illustrating an embodiment of cloning asnapshot tree of a tree data structure. In some embodiments, tree datastructure 250 may be created by a file system manager, such as filesystem manager 105 of secondary storage system 104. The file system dataof a primary system, such as primary system 102, may be backed up to asecondary storage system, such as secondary storage system 104. Asubsequent backup snapshot may correspond to a full backup snapshot oran incremental backup snapshot. The manner in which the file system datacorresponding to the subsequent backup snapshot is stored in secondarystorage system 104 may be represented by a tree data structure. The treedata structure corresponding to the subsequent backup snapshot iscreated by cloning a snapshot tree associated with a last backup.

In the example shown, tree data structure 250 includes a snapshot treethat is comprised of root node 204, intermediate nodes 212, 214, andleaf nodes 222, 224, 226, 228, and 230. Tree data structure 250 may be abackup snapshot of file system data at a particular point in time t+n.The tree data structure allows a chain of snapshot trees to be linkedtogether. Each time a backup snapshot is performed, a root node of thesnapshot tree may be linked to one or more intermediate nodes associatedwith a previous snapshot tree. In the example shown, the snapshot treeat time t+n is linked to the snapshot tree at time t. To create asnapshot tree of the file system data at time t+n, a copy of theprevious root node is created. The root node copy includes the same setof pointers as the original node. However, the root node copy mayinclude a different NodeID and a different TreeID. The TreeID is theidentifier associated with a view. Root node 204 is associated with thesnapshot of the file system data at the particular point in time t+n.Root node 202 is associated with the snapshot of the file system data atthe particular point in time t. The TreeID of a root node indicates asnapshot with which the root node is associated. For example, root node202 with a TreeID of “1” is associated with a first backup snapshot androot node 204 with a TreeID of “2” is associated with a second backupsnapshot.

In the example shown, root node 204 is a copy of root node 202. Similarto root node 202, root node 204 includes the same pointers as root node202, except that root node 204 includes a different node identifier anda different view identifier. Root node 204 includes a first set ofpointers to intermediate node 212. The first set of pointers associatedwith a data key k less than or equal to the node key (e.g., “1,” “2,” or“3”) indicates that traversing tree data structure 250 from root node204 to intermediate node 212 will lead to a leaf node with a data key of“1,” “2,” or “3.” Root node 204 includes a second set of pointers tointermediate node 214. The second set of pointers associated with a datakey k greater than the node key indicates that traversing tree datastructure 250 from root node 204 to intermediate node 214 will lead to aleaf node with a data key of “4” or “5.” Root node 204 includes a NodeIDof “R2” and a TreeID of“2.” The NodeID identifies the name of the node.The TreeID identifies the snapshot with which the node is associated.

FIG. 2C is a block diagram illustrating an embodiment of modifying asnapshot tree. In the example shown, tree data structure 255 may bemodified by a file system manager, such as file system manager 105. Treedata structure 255 may be a current view of the file system data at timet+n. A current view may still accept one or more changes to the data.Because a snapshot represents a perspective of the file system metadatathat is “frozen” in time, one or more copies of one or more nodesaffected by a change to file system metadata are made. Tree datastructure 255 may include a root node that corresponds to a version of asnapshot tree.

In the example shown, the value “DATA4” has been modified to be“DATA4′.” At time t+n, the file system manager starts at root node 204because that is the root node associated with snapshot tree at time t+n.The value “DATA4” is associated with the data key “4.” The file systemmanager traverses tree data structure 255 from root node 204 until itreaches a target node, in this example, leaf node 228. The file systemmanager compares the TreeID at each intermediate node and leaf node withthe TreeID of the root node. In the event the TreeID of a node matchesthe TreeID of the root node, the file system manager proceeds to thenext node. In the event the TreeID of a node does not match the TreeIDof the root node, a shadow copy of the node with the non-matching TreeIDis made. For example, to reach a leaf node with a data key of “4,” thefile system manager begins at root node 204 and proceeds to intermediatenode 214. The file system manager compares the TreeID of intermediatenode 214 with the TreeID of root node 204 (the identifier associatedwith a snapshot at time t=t+n), determines that the TreeID ofintermediate node 214 does not match the TreeID of root node 204, andcreates a copy of intermediate node 214. The intermediate node copy 216includes the same set of pointers as intermediate node 214, but includesa TreeID of “2” to match the TreeID of root node 204. The file systemmanager updates a set of pointers of root node 204 to point tointermediate node 216 instead of pointing to intermediate node 214. Thefile system manager traverses tree data structure 255 from intermediatenode 216 to leaf node 228, determines that the TreeID of leaf node 228does not match the TreeID of root node 204, and creates a copy of leafnode 228. Leaf node copy 232 stores the modified value “DATA4′” andincludes the same TreeID as root node 204. The file system managerupdates a pointer of intermediate node 216 to point to leaf node 232instead of pointing to leaf node 228.

FIG. 2D is a block diagram illustrating an embodiment of a modifiedsnapshot tree of a tree data structure. The tree data structure 255shown in FIG. 2D illustrates a result of the modifications made to treedata structure 255 as described with respect to FIG. 2C.

FIG. 3A is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time. In the example shown, treedata structure 300 is a snapshot tree at time t=1. At t=1, tree datastructure 300 includes a root node 302, intermediate nodes 312, 314, andleaf nodes 322, 324, 326, 328, 330. At t=1, tree data structure 300 issimilar to the tree data structure 200 shown in FIG. 2A. Tree datastructure 300 may correspond to a version of a snapshot tree.

A full backup snapshot or an incremental backup snapshot of the snapshottree at time t=1 may be performed and stored on the storage system. Thefull backup snapshot may provide a complete view of the tree datastructure at a particular point in time, that is, the full backupsnapshot stores all of the nodes associated with a snapshot at theparticular moment in time. For example, a full backup snapshot at timet=1 would include root node 302, intermediate nodes 312, 314, and leafnodes 322, 324, 326, 328, and 330. An incremental backup snapshot mayprovide a partial view of the tree data structure at a particular time.However, in this instance, an incremental backup snapshot at time t=1would also include root node 302, intermediate nodes 312, 314, and leafnodes 322, 324, 326, 328, and 330 because those nodes have not beenpreviously stored.

FIG. 3B is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time. The tree data structure allowsa chain of snapshot trees to be linked together. Each time a backupsnapshot is performed, a root node of the snapshot tree may be linked toone or more intermediate nodes associated with a previous snapshot tree.In the example shown, the snapshot tree at time t=2 is linked to thesnapshot tree at time t=1. At t=2, the snapshot tree includes root node304, intermediate nodes 312, 316, and leaf nodes 322, 324, 326, 330,332. Root node 302 is associated with a snapshot at time t=1 and rootnode 304 is associated with a snapshot at time t=2. At t=2, the treedata structure 350 is similar to the tree data structure 255 shown inFIG. 2D. The snapshot tree at time t=2 is a modified version of thesnapshot tree at time t=1 (i.e., the value of “DATA4” has been modifiedto be “DATA4′”). The snapshot at t=2 may correspond to a version of asnapshot tree.

A full backup snapshot or an incremental backup snapshot of the snapshottree at t=2 may be performed and stored on the storage system. Thesnapshot trees corresponding to a full backup snapshot or incrementalbackup snapshot may provide a complete view of the file system data onthe primary system at a particular point in time to which the snapshottree is associated. For example, the snapshot tree at time t=2 wouldinclude root node 304, intermediate nodes 312, 316, and leaf nodes 322,324, 326, 330, 332. This is a representation of the file system data ofthe primary system at t=2. The snapshot tree at time t 1 includes rootnode 302, intermediate nodes 312, 314, and leaf nodes 322, 324, 326,328, 330. This is a representation of the file system data of theprimary system at t=1. Each snapshot tree is a fully hydrated snapshottree because each snapshot tree provides a complete view of the filesystem data at the time to which the snapshot tree corresponds.

FIG. 3C is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time. In the example shown, treedata structure 380 includes a snapshot tree at time t=3. The tree datastructure allows a chain of snapshot trees to be linked together. Eachtime a backup snapshot is performed, a root node of the snapshot treemay be linked to one or more intermediate nodes associated with aprevious snapshot tree. In the example shown, the snapshot tree at t=3is linked to the snapshot trees at t=1, 2. At t=3, the snapshot treeincludes root node 306, intermediate nodes 312, 318, and leaf nodes 322,324, 326, 330, 334. Root node 302 is associated with a backup snapshotat time t=1, root node 304 is associated with a backup snapshot at timet=2, and root node 306 is associated with a backup snapshot at time t=3.The snapshot tree at t 3 is a modified version of the backup snapshot att=2 (i.e., the value of “DATA4′” has been modified to be “DATA4″”). Thesnapshot tree at t=3 may correspond to a version of a snapshot tree.

FIG. 3D is a block diagram illustrating an embodiment of a tree datastructure at a particular moment in time. In the example shown, treedata structure 390 includes a snapshot tree at time t=4. The tree datastructure allows a chain of snapshot trees to be linked together. In theexample shown, the snapshot tree at time t=4 is linked to the snapshottrees at times t=1, 2, 3. At t=4, the snapshot tree includes root node308, intermediate nodes 312, 318, and leaf nodes 322, 324, 326, 330,334. Root node 302 is associated with a snapshot at time t=1, root node304 is associated with a snapshot at time t=2, root node 306 isassociated with a snapshot at time t=3, and root node 308 is associatedwith a snapshot at time t=4. The snapshot tree at time t=4 maycorrespond to a version of a snapshot tree.

As seen in FIGS. 3B-3D, each snapshot tree builds off of a previoussnapshot tree, that is, a chain of snapshot trees exists. Each snapshottree is associated with a backup snapshot of the file system data. Asmore and more backup snapshots are created and linked, this may requirea lot of storage to store the backup snapshots. To reduce the amount ofstorage needed to store the backup snapshots, a policy may indicate thatafter a full backup snapshot is performed at a particular point in time,one or more previous backup snapshots may be deleted from the storagesystem. In some embodiments, the one or more previous backup snapshotsare deleted after a condition of a retention time policy has passed.

FIG. 4A is a block diagram illustrating an embodiment of archive data. Abackup snapshot is the state of a system at a particular moment in time.A backup snapshot may be stored locally at a storage system, such assecondary storage system 104. A backup snapshot allows the state of asystem to be rolled back to a moment in time for which a backup snapshotis stored. A system may store a large number of backup snapshots (e.g.,thousands, millions). Each backup snapshot may require a significantamount of storage (e.g., GBs, TBs, PBs, etc.). In some instances, it maybe desirable to archive a backup snapshot to a remote storage location,such as cloud 121. For example, one or more older backup snapshots maybe archived to a remote storage location for long-term retention. One ormore backup snapshots may be archived to a remote storage location fordata recovery purposes (e.g., other storage systems may access the dataassociated with a backup snapshot in the event a storage location thatlocally stores the snapshot goes offline). One or more backup snapshotsmay be archived to a remote storage location to handle spikes in storagedemand. One or more backup snapshots that include cold data (i.e., datathat is not accessed frequently) may be archived to a remote storagelocation to free up local storage for one or more snapshots that includehot data (i.e., data that is accessed frequently).

The file system data associated with a backup snapshot may be archivedto a remote storage location. An archive policy may indicate that a fullsnapshot archive of a backup snapshot or an incremental snapshot archiveof the backup snapshot is to be performed and stored on a remote storagelocation. A full snapshot archive includes a complete view of a snapshottree at a particular moment in time. A full snapshot archive includes aroot node associated with the view at the particular moment in time andany intermediate nodes and/or leaf nodes associated with the root node.A full snapshot archive does not include a node of a previous version ofthe snapshot tree if the node is not pointed to by a node associatedwith the view at the particular moment in time. A full snapshot archiveis similar to a full backup snapshot, except that the data associatedwith a full snapshot is stored at a remote location instead of on thestorage system; that is the full snapshot archive includes the datastored in each of the nodes associated with the snapshot tree at theparticular moment in time. For example, a full snapshot archiveassociated with a snapshot at t=3, as depicted in FIG. 3C, includes rootnode 306, intermediate nodes 312, 318, and leaf nodes 322, 324, 326,330, and 334.

An incremental snapshot archive includes a partial view of one versionof a snapshot tree at a particular moment in time. An incrementalsnapshot archive includes a representation of what was not previouslyarchived. An incremental snapshot archive is similar to an incrementalbackup snapshot, except that the data associated with the incrementalbackup snapshot archive is stored at a remote location instead of on thestorage system; that is, the incremental snapshot archive includes thedata stored in the leaf nodes of the snapshot tree that has not beenpreviously archived. For example, an incremental snapshot archiveassociated with a snapshot at t=3, as depicted in FIG. 3C, includes rootnode 306, intermediate node 318, and leaf node 334. The incrementalsnapshot archive at t=3 does not include root nodes 302, 304,intermediate nodes 312, 314, 316, or leaf nodes 322, 324, 326, 328, 330,332 because those nodes were previously archived.

A full snapshot archive may be performed based on one or more policiesassociated with a backup storage system. For example, a full snapshotarchive may be performed on a periodic basis (e.g., every X day(s),every Y week(s), every Z month(s), etc.), upon a threshold size of byteschanging from the previous full snapshot, after a threshold number ofincremental snapshot archives have been performed, etc. A policy mayindicate that an incremental snapshot archive is to be performed on amore frequent basis than a full snapshot archive. The full snapshotarchive and incremental snapshot archives are associated with a backupsnapshot corresponding to a state of file system data at a particularmoment in time. For example, archive data 400 is associated with thesnapshot tree at time t=1, archive data 450 is associated with thesnapshot tree at time t=2, and archive data 480 is associated with thesnapshot tree at time t=3. As seen in FIGS. 4A-4C, each snapshot archivebuilds off of a previous snapshot archive, that is, a block ofserialized data includes a file offset to a block associated withpreviously serialized data. As more and more archives are created, thismay require a lot of storage to store the archives. To reduce the amountof storage needed to store the archives, a policy may indicate thatafter a full snapshot archive, such as depicted in FIG. 4D, is performedat a particular point in time, one or more previous snapshot archives(i.e., archives 400, 450, 480) may be deleted from the remote storagelocation. In some embodiments, the one or more previous snapshotarchives are deleted after a condition of retention time policy haspassed.

In the example shown, archive data 400 can be archived by a storagesystem, such as secondary storage system 104, to a remote storagelocation, such as cloud 121. Archive data 400 may be stored in cloudobject storage of cloud storage, such as cloud object storage 124 ofcloud 121. In the example shown, archive data 400 includes file systemdata 451 and serialized snapshot tree data 461. In the example shown,archive data 400 is a file representation of a backup snapshot of thesnapshot tree at a particular moment in time, t=1. Archive data 400stores a full backup snapshot of the snapshot tree at time t=1. A fullsnapshot archive includes a complete view of the nodes of the snapshottree at a particular moment in time (i.e., all nodes associated with aroot node of the snapshot tree) and the data referenced or stored ineach of the leaf nodes of the snapshot tree. For example, a leaf nodemay include a pointer to a storage location of a value. A full snapshotarchive is independent on its own and does not refer back to one or moreprevious snapshot archives.

File system data of archive data that stores a full backup snapshot of asnapshot tree associated with a particular view includes all of the datastored in or referenced by the one or more leaf nodes of a backupsnapshot, regardless of when a leaf node was created (i.e., the snapshotmay include leaf nodes associated with previous snapshots). In theexample shown, file system data 451 corresponds to data stored in theleaf nodes of the snapshot tree at time t=1. Since archive data 400includes a full backup snapshot of the snapshot tree at t=1, file systemdata 451 includes the data stored in or referenced by leaf nodes 322,324, 326, 328, and 330 in FIG. 3A, that is, file system data 451includes “DATA1,” “DATA2,” “DATA3,” “DATA4,” and “DATA5.” File systemdata 451 may be archived from a storage system, such as a secondarystorage system 104, to a remote storage location, such as cloud 121. Insome embodiments, the file system data is the data (e.g., data blocks ofa file, data segments of a file) for a distributed file system. Filesystem data may be stored as a flat set of data. In some embodiments,file system data 451 stores all the data blocks associated with leafnodes of a snapshot tree. In other embodiments, file system data 451stores all 1s and 0s of file data blocks associated with leaf nodes of asnapshot tree. In some embodiments, file system data 451 stores aplurality of file data blocks in a single block of file system data 451.In some embodiments, the file system data includes file system metadata,such as file size, directory structure, file permissions, physicalstorage locations of the files, etc.

A serialized snapshot tree data is configured to store the structure ofthe snapshot tree associated with the file system data as a flat set ofdata that is comprised of one or more blocks. Each block of the flat setof data corresponds to a node of the snapshot tree. A block may containa file offset. A file offset represents a pointer of a snapshot tree.Because some archive systems cannot store pointers, a file offset isused in place of pointers. The file offset may be to another block ofthe serialized snapshot tree data. The file offset may be to anotherblock of a different serialized snapshot tree data.

In the example shown, serialized snapshot tree data 461 corresponds to asnapshot tree at time t=1. Serialized snapshot tree data 461 iscomprised of a plurality of blocks. Each block corresponds to one of thesnapshot tree nodes. For example, blocks 422, 424, 426, 428, 430, 412,414, and 402 correspond to nodes 322, 324, 326, 328, 330, 312, 314, and302, respectively, of the snapshot tree at t=1 in FIG. 3A.

Block 402 corresponds to root node 302. Because root node 302 includespointers to intermediate nodes 312 and 314, block 402 includes fileoffsets to blocks 412 and 414. Blocks 412 and 414 correspond tointermediate nodes 312 and 314, respectively. Because intermediate node312 includes pointers to leaf nodes 322, 324, and 326, block 412includes file offsets to blocks 422, 424, and 426. The file offsetscorrespond to the pointers of a snapshot tree. Similarly, block 414includes file offsets to blocks 428, 430 because intermediate node 314includes pointers to leaf nodes 328, 330.

Blocks 422, 424, 426, 428, and 430 correspond to the leaf nodes ofsnapshot tree 300 and each include a corresponding file offset to one ormore blocks of the file system data stored in file system data 451. Forexample, block 422 includes an offset to one or more blocks in filesystem data 451 that store the value of L1. Similarly, blocks 424, 426,428, 430 include corresponding offsets to one or more blocks in filesystem data 451 that store the value of L2, L3, L4, and L5,respectively.

FIG. 4B is a block diagram illustrating an embodiment of archive data.In the example shown, archive data 450 can be archived by a storagesystem, such as secondary storage system 104. In the example shown,archive data 450 includes file system data 453 and a serialized snapshottree data 463.

File system data 453 is an incremental snapshot archive of the filesystem data stored in the one or more leaf nodes of a snapshot tree. Anincremental snapshot archive may include changes to the data of asnapshot tree since a last snapshot archive (e.g., new data or modifieddata). For example, file system data 453 may include one or more valuesstored in or referenced by leaf nodes of the snapshot tree at time t=2that were not previously archived. File system data 453 may be stored asa flat set of data. In some embodiments, file system data 453 stores alldata blocks associated with leaf nodes of a snapshot tree that were notpreviously archived. In other embodiments, file system data 453 storesthe corresponding 1s and 0s of file data blocks associated with leafnodes of a snapshot tree that were not previously archived. In someembodiments, file system data 453 stores a plurality of file data blocksin a single block of file system data 453. In some embodiments, the filesystem data includes file system metadata, such as file size, directorystructure, file permissions, physical storage locations of the files,etc.

Serialized snapshot tree data 463 is a serialized version of one or morenodes of the snapshot tree at time t=2 and is represented as a flat setof data that is comprised of one or more blocks. Each block of the flatset of data corresponds to a node of the snapshot tree. Serializedsnapshot tree data 463 includes a serialized representation of one ormore changes to a snapshot tree (e.g., new node, modified node, deletednode) since a previous snapshot. Serialized snapshot tree data mayinclude a block for each root node of a snapshot tree.

To determine whether a node should be included in a serialized snapshottree data, a file system manager starts at the root node associated witha snapshot view and traverses the snapshot tree. At each node of thesnapshot tree, the file system manager determines whether thatparticular node existed and is the same as the one in the previoussnapshot tree. In the event the node didn't exist in the previoussnapshot tree or is different when compared to the corresponding node inthe previous snapshot tree, a block corresponding to the node isincluded in serialized snapshot tree data. In the event the node isdetermined to have existed in the previous snapshot tree and is also thesame as the corresponding node in the previous snapshot tree, a blockcorresponding to the node is not included in the serialized snapshottree data because a previous serialized snapshot tree data alreadyincludes a block corresponding to the node. Instead, a file offset tothe block of the previous serialized snapshot tree data may be includedin one or more of the blocks in the serialized snapshot tree data.

For example, to create a snapshot at t=2, root node 304 was added. Thesnapshot tree at t=2 indicates that the value of “DATA4” has beenmodified to be “DATA4′.” Intermediate node 316 and leaf node 332 wereadded to the snapshot tree to ensure that each node along this path hasa TreeID of “2.”

In the example shown, serialized snapshot tree data 463 corresponds tothe root nodes of the snapshot tree at t=2 and the new nodes of snapshottree at t=2. Serialized snapshot tree data 463 is comprised of aplurality of blocks. Each block corresponds to one of the nodesassociated with the snapshot tree at t 2. For example, blocks 432, 416,404 correspond to nodes 332, 316, 304, respectively. In otherembodiments, serialized snapshot tree data corresponding to anincremental backup includes the root node associated with a snapshotview.

Block 404 corresponds to root node 304. Because root node 304 includes apointer to intermediate node 312, block 404 includes a file offset toblock 412 of serialized snapshot tree data 461. Previously storedserialized snapshot tree data 461 already includes block 412 thatcorresponds to intermediate node 312. A file offset to a previouslystored serialized snapshot tree data is used to save memory and preventstoring duplicative data. Root node 304 also includes a pointer tointermediate node 316. Similarly, block 404 also includes a file offsetto block 416, which corresponds to intermediate node 316.

Block 416 corresponds to intermediate node 316. Intermediate node 316includes pointers to leaf nodes 330, 332. The value of leaf node 330 hasnot changed and was previously stored in file system metadata 451. Tosave memory and prevent storing duplicative data, block 416 includes afile offset to block 430 of serialized snapshot tree data 461. Block 416also includes a file offset to block 432. Block 432 corresponds to leafnode 332. Intermediate node 316 is a new node because snapshot tree 300did not include intermediate node 316. Thus, serialized snapshot treedata 463 includes a block that corresponds to intermediate node 316.

Block 432 corresponds to leaf node 332 of snapshot tree 350. Leaf node332 is a new node because snapshot tree 300 did not include leaf node332. Thus, serialized snapshot tree data 463 includes a block thatcorresponds to leaf node 332. Block 432 includes a file offset to one ormore blocks in file system data 453 that store the value of leaf node332.

FIG. 4C is a block diagram illustrating an embodiment of archive data.In the example shown, archive data 480 can be archived by a system, suchas secondary storage system 104. In the example shown, archive data 480includes file system data 455 and a serialized snapshot tree data 465.

File system data 455 is an incremental snapshot archive of the filesystem data stored in or referenced by the one or more leaf nodes of asnapshot tree. For example, file system data 455 may include one or morevalues of the snapshot tree at time t=3 that were not previouslyarchived. File system data 455 may be stored as a flat set of data. Insome embodiments, file system data 455 stores all data blocks associatedwith leaf nodes of a snapshot tree that were not previously archived. Inother embodiments, file system data 455 stores the corresponding 1s and0s of file data blocks associated with leaf nodes of a snapshot treethat were not previously archived. In some embodiments, file system data455 stores a plurality of file data blocks in a single block of filesystem data 455. In some embodiments, the file system data includes filesystem metadata, such as file size, directory structure, filepermissions, physical storage locations of the files, etc.

Serialized snapshot tree data 465 is a serialized version of one or morenodes of the snapshot tree at time t=3 and is represented as a flat setof data that is comprised of one or more blocks. Each block of the flatset of data corresponds to a node of the snapshot tree. To create asnapshot at t=3, root node 306 was added. The snapshot tree indicatesthat the value of “DATA4′” has been modified to be “DATA4″.”Intermediate node 318 and leaf node 334 were added to the snapshot treeat t=3 to ensure that each node along this path has a TreeID of “3.”

In the example shown, serialized snapshot tree data 465 corresponds toroot nodes of the snapshot tree at time t=3 and the new nodes of thesnapshot tree at time t=3. Serialized snapshot tree data 465 iscomprised of a plurality of blocks. Each block corresponds to one of thenodes of the snapshot tree at time t=3. For example, blocks 434, 418,406 correspond to nodes 334, 318, 306, respectively.

Block 406 corresponds to root node 306. Because root node 306 includes apointer to intermediate node 312, block 406 includes a file offset toblock 412 of serialized snapshot tree data 461. Root node 306 includes apointer to intermediate node 318. Similarly, block 406 includes a fileoffset to block 418, which corresponds to intermediate node 318.

Block 418 corresponds to intermediate node 318. Intermediate node 318includes a pointer to leaf nodes 330, 334. The value of leaf node 330has not changed and was previously stored in file system metadata 451.To save memory and prevent storing duplicative data, block 418 includesa file offset to block 430 of serialized snapshot tree data 461. Block418 also includes a file offset to block 434. Block 434 corresponds toleaf node 334. Intermediate node 318 is a new node because snapshot tree350 did not include intermediate node 318. Thus, archive data 480includes a block that corresponds to intermediate node 318.

Block 434 corresponds to leaf node 334 of snapshot tree 380. Leaf node334 is a new node because snapshot tree 350 did not include leaf node334 at t=2. Thus, archive data 480 includes a block that corresponds toleaf node 334. Block 434 includes a file offset to a block of filesystem metadata 455 that stores the value of leaf node 334.

FIG. 4D is a block diagram illustrating an embodiment of archive data.In the example shown, archive data 490 can be archived by a storagesystem, such as secondary storage system 104. In the example shown,archive data 490 includes file system data 457 and a serialized snapshottree data 467. In the example shown, archive data 490 is a filerepresentation of snapshot tree at time t=4. Archive data 490 stores afull snapshot of the snapshot tree at time t=4. A full snapshot archiveincludes a representation of all of the nodes of a snapshot of asnapshot tree at a particular moment in time (i.e., all nodes associatedwith a root node of the snapshot tree) and the data stored in each ofthe leaf nodes of the snapshot tree.

In the example shown, serialized snapshot tree data 467 corresponds tothe snapshot tree at t=4. Serialized snapshot tree data 467 is comprisedof a plurality of blocks. Each block corresponds to one of the snapshottree nodes. For example, blocks 422, 424, 426, 434, 430, 412, 418, and408 correspond to nodes 322, 324, 326, 334, 330, 312, 318, and 308,respectively, of the snapshot tree at time t=4.

Block 408 corresponds to root node 308. Because root node 308 includespointers to intermediate nodes 312 and 318, block 408 includes fileoffsets to blocks 412 and 418. Blocks 412 and 418 correspond tointermediate nodes 312 and 318, respectively. Because intermediate node312 includes pointers to leaf nodes 322, 324, and 326, block 412includes file offsets to blocks 422, 424, and 426. The file offsetscorrespond to the pointers of a snapshot tree. Similarly, block 418includes file offsets to blocks 434, 430 because intermediate node 318includes pointers to leaf nodes 334, 330.

Blocks 422, 424, 426, 434, and 430 correspond to the leaf nodes ofsnapshot tree 390 and each include a corresponding file offset to one ormore blocks of the file system data stored in file system data 457. Forexample, block 422 includes an offset to one or more blocks in filesystem data 457 that stores the value of L1. Similarly, blocks 424, 426,434, 430 include corresponding offsets to one or more blocks in filesystem data 457 that store the value of L2, L3, L7, and L5,respectively.

As seen in FIGS. 4A-4C, a serialized snapshot tree data may be linkedwith a previous serialized snapshot tree data. As more and more backupsnapshots are archived, this may require a lot of storage to archive theserialized snapshot tree data. To reduce the amount of storage needed tostore the archives, a policy may indicate that after a full snapshotarchive, such as archive 490, is performed at a particular point intime, one or more previous archives may be deleted from cloud storage,i.e., archive data 400, 450, 480. For example, archive data 400, 450,480 may be deleted after archive data 490 is archived. In someembodiments, the one or more previous snapshot archives are deletedafter a condition associated with a retention time policy has passed.For example, a policy may indicate that data is to be archived for aperiod of thirty days.

FIG. 5 is a flow chart illustrating an embodiment of archiving data. Inthe example shown, process 500 may be implemented by a storage system,such as secondary storage system 104. In some embodiments, process 500is a full snapshot archive. In other embodiments, process 500 is anincremental snapshot archive.

At 502, is it is determined that file system data is to be archived. Abackup snapshot is the state of a system at a particular moment in time.A backup snapshot may be stored locally at a storage system, such assecondary storage system 104. A backup snapshot allows the state of asystem to be rolled back to a moment in time for which a snapshot isstored. A system may store a large number of backup snapshots (e.g.,thousands, millions). Each backup snapshot may require a significantamount of storage (e.g., GBs, TBs, PBs, etc.). In some instances, it maybe desirable to archive a backup snapshot to a remote storage location,such as cloud 121. The file system data associated with a backupsnapshot may be archived to a remote storage location. An archive policymay indicate that a full snapshot archive of a snapshot or anincremental snapshot archive of the backup snapshot is to be performedand stored on a remote storage location. A full snapshot archiveincludes a complete view of one version of a snapshot tree at aparticular moment in time. A full snapshot archive includes a root nodeassociated with the view at the particular moment in time and anyintermediate nodes and/or leaf nodes associated with the root node. Afull snapshot archive does not include a node of a previous version ofthe snapshot tree if the node is not pointed to by a node associatedwith the view at the particular moment in time. A full snapshot archiveis similar to a full backup snapshot, except that the data associatedwith the full backup snapshot is stored at a remote location instead ofon the storage system; that is, the full snapshot archive includes thedata stored in each of the nodes associated with the snapshot tree atthe particular moment in time. For example, a full snapshot archiveassociated with a snapshot at t=3, as depicted in FIG. 3C, includes rootnode 306, intermediate nodes 312, 318, and leaf nodes 322, 324, 326,330, and 334.

An incremental snapshot archive includes a partial view of one versionof a snapshot tree at a particular moment in time. An incrementalsnapshot archive includes a representation of what was not previouslyarchived. An incremental snapshot archive is similar to an incrementalsnapshot, except that the data associated with the incremental snapshotarchive is stored at a remote location instead of on the storage system;that is, the incremental snapshot archive includes the data stored inthe leaf nodes of the snapshot tree that have not been previouslyarchived. For example, an incremental snapshot archive associated with abackup snapshot at t=3, as depicted in FIG. 3C, includes root node 306,intermediate node 318, and leaf node 334. The incremental snapshotarchive at t=3 does not include root nodes 302, 304, intermediates nodes312, 314, 316, or leaf nodes 322, 324, 326, 328, 330 332 because thosenodes were previously archived.

In some embodiments, the filesystem data is archived according to anarchive policy. For example, an archive policy may indicate that a fullsnapshot archive is to be performed on a periodic basis (e.g., every Whour(s), every X day(s), every Y week(s), every Z month(s), etc.). Anarchive policy may indicate that a full snapshot archive is to beperformed each time a full backup snapshot is performed. In someembodiments, an archive policy may indicate that one or more previoussnapshot archives are to be deleted after a full backup snapshot archiveis performed. In some embodiments, an archive policy may indicate thatone or more file system data files and corresponding serialized snapshottree data are to be deleted after a full snapshot archive is performed.An archive policy may indicate that an incremental snapshot archive isto be performed on a periodic basis (e.g., every W hour(s), every Xday(s), every Y week(s), every Z month(s), etc.). An archive policy mayindicate that an incremental snapshot archive is to be performed eachtime an incremental snapshot is performed. An archive policy mayindicate that an incremental snapshot archive is to be performed on amore frequent basis than a full snapshot archive. The full snapshotarchive and incremental snapshot archives are associated with a backupsnapshot at a particular moment in time. For example, archive data 400is associated with the snapshot tree at time t=1, archive data 450 isassociated with the snapshot tree at time t=2, and archive data 480 isassociated with the snapshot tree at time t=3. As seen in FIGS. 4A-4C,each snapshot archive builds off of a previous snapshot archive, thatis, a block of serialized data includes a file offset to a blockassociated with previously serialized data. As more and more archivesare created, this may require a lot of storage to store the archives. Toreduce the amount of storage needed to store the archives, a policy mayindicate that after a full snapshot archive, such as depicted in FIG.4D, is performed at a particular point in time, one or more previoussnapshot archives (i.e., archives 400, 450, 480) may be deleted from theremote storage location. In some embodiments, the one or more previoussnapshot archives are deleted after a condition of retention time policyhas passed.

At 504, a snapshot tree associated with a view is serialized intoserialized snapshot tree data and file system data associated with theview is serialized into serialized file system data. Serializing thesnapshot tree into serialized snapshot tree data creates a flat set ofdata that represents the snapshot tree. In some embodiments, the filesystem data may be stored in a tree data structure, such as the treedata structure depicted in FIG. 2E. Serializing the file system datainto serialized file system data creates a flat set of data thatrepresents the file system data. The snapshot tree and the file systemdata are serialized into flat sets of data because a remote location maybe incapable of storing a tree data structure.

The serialized snapshot tree data, i.e., a flat set of data, iscomprised of one or more blocks. The serialized snapshot tree is arepresentation of a snapshot tree in block form. Each block of theserialized snapshot tree data corresponds to a node of a snapshot tree.Instead of a node having one or more pointers to one or more othernodes, a block of the serialized snapshot tree may include one or morefile offsets to one or more other blocks. The file offsets represent thepointers of a snapshot tree. A block may include a file offset toanother block in the serialized snapshot tree data. A block may includea file offset to another block in a previously serialized snapshot treedata. For example, a snapshot tree node may include a pointer to a nodeassociated with a previous snapshot tree. A block that corresponds tothe snapshot tree node may include a file offset to the block of apreviously serialized snapshot tree data block that corresponds to thenode associated with the previous snapshot tree. The snapshot tree nodemay also include a pointer to a node associated with the currentsnapshot tree. A block that corresponds to the snapshot tree node mayinclude a file offset to the block of the current serialized snapshottree data that corresponds to the node associated with the currentsnapshot tree.

The serialized file system data, i.e., a flat set of data, is comprisedof one or more blocks. Each block of the serialized file system datacorresponds to a data block or data segment of the file system data.

In some embodiments, a full snapshot is performed and the serializedsnapshot tree data includes a plurality of blocks that correspond to theplurality of nodes of the snapshot tree. In some embodiments, one ormore backup snapshots performed before the full backup snapshot aredeleted. In other embodiments, an incremental backup snapshot isperformed and the serialized snapshot tree data includes a plurality ofblocks that correspond to the one or more root nodes and the one or morenodes that have been added to a snapshot tree since a previous archive.In some embodiments, an incremental backup snapshot is performed for aplurality of different snapshot trees and the corresponding serializedsnapshot tree data includes file blocks corresponding to the pluralityof different snapshot trees. In some embodiments, a serialized snapshottree data combines the plurality of blocks that correspond to the one ormore root nodes and the one or more nodes that have been added to asnapshot tree since a previous archive with one or more blocks from oneor more previous archives.

At 506, the serialized snapshot tree data and serialized file systemdata are archived. The serialized snapshot tree data and serialized filesystem data may be archived, to a remote location, such as cloud 121.Within cloud storage, the snapshot archive comprising the serializedsnapshot tree data and serialized file system data may be stored in acloud object storage location, such as cloud object storage 124. Thefile system data is comprised of data blocks of a file and/or datasegments of a file, and may be stored as a set of flat data. In someembodiments, the file system data is a full snapshot archive of the filesystem data stored in or referenced by the one or more leaf nodes of asnapshot tree. Each of the data blocks/segments comprising the filesystem data stored in the one or more leaf nodes of a snapshot tree maybe stored in the set of flat data. In some embodiments, the file systemdata is an incremental snapshot archive of the file system data storedin or referenced by the one or more leaf nodes of a snapshot tree. Theincremental snapshot archive may include changes to the data of asnapshot tree since a last snapshot archive (e.g., new data or modifieddata). Each of the data blocks/segments comprising the changes to thedata of a snapshot tree since a last snapshot archive may be stored inthe set of flat data. In some embodiments, the serialized snapshot treedata and file system data are archived to remote storage (e.g., cloud,cluster storage system, etc.). The file system data may be stored in oneor more files. File system metadata may be stored in one or moreseparate files. The file system metadata may include a reference to afile system data file and vice versa.

FIG. 6 is a flow chart illustrating an embodiment of a process forrestoring data. In the example shown, process 600 may be performed by acloud portal, such as cloud portal 123.

At 602, an indication that a secondary storage system is offline isreceived. A secondary storage system may be coupled to a primary systemand configured to receive a backup snapshot from the primary system. Thebackup snapshot is comprised of one or more data blocks. In response toreceiving the backup snapshot, the secondary storage system isconfigured to store the one or more data blocks and organize the one ormore data blocks using a tree data structure. The secondary storagesystem is configured to store a plurality of backup snapshots associatedwith the primary system. The secondary storage system is configured toarchive to cloud storage one or more of the backup snapshots based on anarchive policy. The archive policy may indicate that a backup snapshotis to be archived after being stored on the secondary storage system fora threshold duration (e.g., 30 days). The archive policy may indicatethat a full snapshot archive of the file system data and/or anincremental snapshot archive of the file system data is to be performed.

A user associated with the primary system may send a request to thesecondary storage system. The request may be a request to perform abackup snapshot to the secondary storage system. The request may be arequest to restore one or more of the stored backup snapshots. Thesecondary storage system may be unable to satisfy the request for one ormore reasons. For example, the secondary storage system may be offlinedue to a power outage, a plurality of the storage nodes comprising thesecondary storage system have failed, etc.

In the event the secondary storage system is unable to restore to theprimary system one or more of the stored backup snapshots, the primarysystem may provide the user an error message indicating that thesecondary storage system is unable to restore the one or more storedbackup snapshots. In response to receiving the error message, a userassociated with the primary system may log into the cloud portal tostart the cloud instantiation process.

In other embodiments, the secondary storage system may provide aheartbeat signal to the primary system. In the event the primary systemdoes not receive the heartbeat signal within a threshold period, theprimary system is configured to provide to a cloud portal an indicationthat the secondary storage system is offline, which causes the cloudportal to generate a cloud instantiation of the secondary storagesystem.

At 604, a cloud instantiation of the secondary storage system isgenerated. A secondary storage system is comprised of a plurality ofstorage nodes. Each storage node has a particular storage capacity. Acloud portal may provision resources for the cloud instantiation of thesecondary storage system. The cloud instantiation of the secondarystorage system corresponds to a virtual secondary storage cluster. Thevirtual secondary storage cluster may be configured to have the samestorage capacity as the secondary storage system. For example, asecondary storage system may be comprised of three physical storagenodes, each physical storage node having a storage capacity of 10 TB.The cloud instantiation of the secondary storage system may be comprisedof three virtual cloud instances, each virtual cloud instance having astorage capacity of 10 TB. The cloud instantiation of the secondarystorage system may be configured for the public cloud (e.g., Amazon WebServices, Microsoft Azure, Google Cloud, etc.) in which the cloudinstantiation will reside. A user may specify the public cloud in whichthe cloud instantiation will reside. In other embodiments, the virtualsecondary storage cluster is configured to have more storage capacitythan the secondary storage system. In other embodiments, the virtualsecondary storage cluster is configured to have less storage capacitythan the secondary storage system.

In other embodiments, the virtual secondary storage cluster may beconfigured to have a user-specified storage capacity. For example, theuser may request to have 50 TBs of storage. Each virtual cloud instancemay be configured to have a default storage capacity (e.g., 10 TB). Tosatisfy the user's request, the cloud instantiation of the secondarystorage system may be comprised of five virtual cloud instances, eachvirtual cloud instance having a storage capacity of 10 TB. In otherembodiments, the cloud instantiation of the secondary storage system isconfigured to have a default storage capacity (e.g., a virtual secondarystorage cluster comprised of three virtual cloud instances, each havinga storage capacity of 10 TB).

At 606, a user is authenticated. A user associated with the cloudinstantiation of the secondary storage system may log into a userinterface of the cloud instantiation. A cloud storage is configured tostore a plurality of snapshot archives. An enterprise may be associatedwith one or more data centers. Each data center may have a correspondingsecondary storage system. The corresponding secondary storage systemsmay be configured to archive corresponding backup snapshots to cloudstorage based on one or more archive policies. A user associated withthe enterprise may be permitted to access a snapshot archive and requesta snapshot archive to be restored to one of the one or more data centersassociated with the enterprise. In other embodiments, the user isassociated with one of the enterprise's data centers. The user may bepermitted to access snapshot archives specific to that particular datacenter and restore to a primary system of the particular data center orthe secondary storage system of the particular data center, the snapshotarchives specific to that particular data center.

A plurality of enterprises may have corresponding secondary storagesystems. Each of the corresponding secondary storage systems may beconfigured to archive an enterprises' backup snapshots. A user may bepermitted to access a subset of the snapshot archives. For example, auser associated with a first enterprise may be permitted to accesssnapshot archives associated with the first enterprise, but notpermitted to access snapshot archives associated with a secondenterprise. Similarly, a user associated with a second enterprise may bepermitted to access snapshot archives associated with the secondenterprise, but not permitted to access snapshot archives associatedwith the first enterprise.

The cloud portal may be configured to request the user to provide acredential that indicates the user is permitted to access the one ormore snapshot archives associated with an enterprise. The user'scredential may be linked to a subset of the plurality of snapshotarchives. For example, the credential of the user associated with thefirst enterprise is linked to the snapshot archives associated with thefirst enterprise and the credential of the user associated with thesecond enterprise is linked to the snapshot archives associated with thesecond enterprise.

Upon authenticating the user, the user may have access to any of thesnapshot archives included in the subset of snapshot archives.

At 608, an indication of an external target is received. The externaltarget corresponds to a user destination system that will receive thedata associated with a snapshot archive. The user destination system maycorrespond to a primary system of a data center or a secondary storagesystem of the data center. The archive data associated with a secondarystorage system may be encrypted. The indication may include a key todecrypt the archive data.

At 610, the cloud retrieve process is started. A list of one or moresnapshot archives available to be restored may be presented to a uservia a cloud user interface. A request for one or more snapshot archivesis received from the user. The request may include an associateddestination for the file system data associated with the selectedsnapshot archive. The request may specify which secondary storagesystems the user desires to restore (e.g., an enterprise may beassociated with a plurality of secondary storage systems, which snapshotarchives to restore, a date range associated with a snapshot archive torestore, and a format for the file system data associated with thesnapshot archive.)

One or more secondary storage clusters may be virtually rebuilt in thecloud instantiation of the secondary storage system using the one ormore snapshot archives requested by the user. Virtually rebuilding asecondary storage cluster includes reconstituting a tree data structurebased on the one or more requested snapshot archives.

A snapshot archive may correspond to a backup snapshot that was storedon the secondary storage system and archived to cloud storage. In someembodiments, the snapshot archive corresponds to a backup snapshot thatis stored on the secondary storage system (e.g., a duplicate copy of thebackup snapshot). In other embodiments, the snapshot archive correspondsto a backup snapshot that is not stored on the secondary storage system(e.g., the backup snapshot was stored on the secondary storage systempast a retention period, archived to cloud storage, and removed from thesecondary storage system).

The request for one or more snapshot archives may be for the entiresnapshot archive or a portion of the snapshot archive. For example, auser may request to restore an entire snapshot archive to restore theprimary system to a particular moment in time. The user may request torestore a portion of the snapshot archive to restore one or more filesthat are included in the snapshot archive.

At 612, the requested data is provided to the external target. A usermay register and provide an external target for the requested data. Forexample, the user may register a primary system to receive the requesteddata. The user may register another secondary storage system to receivethe requested data, (e.g., a new secondary storage system is broughtonline).

A snapshot archive is comprised of file system data and serializedsnapshot tree data. The cloud instantiation of the secondary storagesystem is configured to reconstitute a snapshot tree associated with thesnapshot archive by deserializing the serialized data file. The cloudinstantiation of the secondary storage system is configured to store thefile system data and the deserialized tree data across the virtual cloudinstances (e.g., the file system data is stored in the cloudinstantiation of the secondary storage system). In some embodiments, thecloud instantiation of the secondary storage system is configured toprovide all of the file system data associated with the snapshotarchive.

In other embodiments, the cloud instantiation of the secondary storagesystem is configured to provide a portion of the file system dataassociated with the snapshot archive. For example, a subset of the filesincluded in the snapshot archive may be requested. The cloudinstantiation of the secondary storage system is configure to traversethe reconstituted snapshot tree and to locate the file system dataassociated with the requested subset of files. Upon location, the cloudinstantiation of the secondary storage system may provide the requesteddata to the primary system associated with the user or to anotherlocation.

FIG. 7 is a flow chart illustrating an embodiment of restoring archiveddata. In the example shown, process 700 may be performed by a storagesystem, such as a cloud instantiation 122 of secondary storage system104. Process 700 may be implemented to perform some or all of steps 610,612 of process 600.

At 702, a request for one or more snapshot archives may be received. Aprimary system may be configured to send a backup snapshot comprisingprimary system file system data to a secondary storage system. Thebackup snapshot is comprised of a plurality of data blocks. In responseto receiving the backup snapshot, the secondary storage system may beconfigured to store the data blocks associated with the backup snapshotand to organize the file system data using a tree data structure, e.g.,a snapshot tree.

The secondary storage system (e.g., a secondary storage cluster) may beconfigured to archive a snapshot tree to a remote storage location, suchas cloud storage. A snapshot archive may include file system data andserialized snapshot tree data. File system data includes one or moredata blocks that comprise the file system data. A snapshot treeassociated with the file system data is serialized into serializedsnapshot tree data. Serializing is a process by which a data file isgenerated to store data in a manner that mimics the structure of a treedata structure. The serialized data file may be encoded in a manner thatallows the serialized data file to be utilized to reconstruct a desiredportion of the tree data structure to obtain a data of interest from theserialized data file without the need to reconstruct the entire treedata structure. A serialized snapshot tree data stores the structure ofthe snapshot tree associated with the file system data as a flat set ofdata that is comprised of one or more blocks. Each block of the flat setof data corresponds to a node of the snapshot tree. The order of thedata blocks of the serialized data file corresponds to an order of thetree data structure. A tree data structure may have a root node, aplurality of intermediate nodes, and a plurality of leaf nodes. Theserialized data file may include blocks corresponding to the root node,then the plurality of intermediate nodes, and then the plurality of leafnodes. For example, a first data block of the serialized data file maycorrespond to a root node of the tree data structure, a second datablock of the serialized data file may correspond to a first intermediatenode of the tree data structure, a third data block of the serializeddata file may correspond to a second intermediate node of the tree datastructure, a fourth data block of the serialized data file maycorrespond to a first leaf node of the tree data structure, . . . and annth data block of the serialized data file may correspond to the nthleaf node of the tree data structure. The blocks that correspond to anintermediate node or a root node may include a file offset to anotherblock of the serialized snapshot tree data or a file offset to anotherblock of a different serialized snapshot tree data, i.e., serializedsnapshot tree data associated with a different snapshot archive. A blockthat corresponds to a leaf node may include a file offset to a portionof the archived file system data.

In some embodiments, the request for one or more snapshot archives isfor a snapshot archive that corresponds to an incremental snapshotarchive. For example, a user may desire to restore one or more filesassociated with a backup snapshot without having to restore all of thefile system data associated with a backup snapshot.

In other embodiments, the request for one or more snapshot archives isfor a snapshot archive that corresponds to a full snapshot archive. Forexample, a user may desire to restore the file system of a primarysystem or other system to a state associated with a full backupsnapshot.

In other embodiments, the request for one or more snapshot archives is asnapshot archive that corresponds to an incremental snapshot archive andone or more other snapshot archives. For example, a user may desire torestore the file system of a primary system or other system to a stateassociated with an incremental backup snapshot. Since the incrementalbackup snapshot includes file system data that was not previously backedup, file system data associated with one or more other backup snapshotsare needed to restore the file system to the state associated with theincremental backup snapshot. The file system data associated with theone or more other backup snapshots is stored in the one or more othersnapshot archives.

In some embodiments, a request for one snapshot archive causes one ormore other snapshot archives associated with the requested snapshotarchive to be requested because the one or more other snapshot archivesare needed to restore the file system of a primary system or othersystem to a particular state.

At 704, the one or more requested snapshot archives are retrieved fromcloud storage. A snapshot archive is comprised of file system dataassociated with the snapshot tree and serialized snapshot tree dataassociated with a snapshot tree. An archive of file system data maycomprise a first set of flat data. An archive of serialized snapshottree data may comprise a second set of flat data. For example, at leasta portion of one or more file system archives 451, 453, 455 of FIGS.4A-4C and at least a portion of one or more archives of serializedsnapshot tree data 461, 463, 465 of FIGS. 4A-4C may be retrieved. Insome embodiments, data corresponding to an incremental snapshot archiveis retrieved. In some embodiments, data corresponding to a full snapshotarchive is retrieved. In some embodiments, data corresponding to a fullsnapshot archive and one or more incremental snapshot archives areretrieved. The archive file system data is stored in the virtual cloudinstances of the cloud instantiation. In some embodiments, the filesystem data is stored in the same manner in the virtual cloud instancesof the cloud instantiation as the physical storage nodes of thesecondary storage system.

At 706, a snapshot tree associated with the one or more retrievedsnapshot archives is reconstituted. A virtual file manager of the cloudinstantiation may virtually rebuild one or more secondary storagesystems by reconstituting a snapshot tree by deserializing serializedsnapshot tree data associated with a snapshot archive. In otherembodiments, the snapshot tree is reconstituted by deserializingserialized snapshot tree data associated with a plurality of snapshotarchives. In some embodiments, the one or more other snapshot archivescomprising serialized snapshot tree data were archived before therequested snapshot archive comprising the serialized snapshot tree data.Reconstituting the structure of a snapshot tree corresponding to therequested snapshot archive includes reading the flat set of dataassociated with the serialized snapshot tree data. The flat set of datamay include blocks of data that correspond to nodes of a snapshot treeand associated file offsets that correspond to pointers of the snapshottree.

For example, for a request associated with snapshot archive 480, thecomplete snapshot tree at t=3 may be reproduced based on serializedsnapshot tree data 465, 463, 461. The virtual file system manager of acloud instantiation may deserialize the serialized snapshot tree data.Root node 306 may be reproduced because serialized snapshot tree data465 includes a block 406 that corresponds to root node 306 of thesnapshot tree, which includes offsets to blocks associated withintermediate nodes 312, 318.

Intermediate node 312 may be reproduced because block 406 includes anoffset to block 412, which corresponds to intermediate node 312. Thedata associated with intermediate node 312 may be determined fromserialized snapshot tree data 461.

Intermediate node 318 may be reproduced because block 406 includes anoffset to block 418, which corresponds to intermediate node 318. Thedata associated with intermediate node 318 may be determined fromserialized snapshot tree data 465.

Leaf node 334 may be reproduced because block 418 includes an offset toblock 434, which corresponds to leaf node 334. The value associated withleaf node 334 may be accessed and reproduced because block 434 includesan offset to one or more blocks of data stored in file system data 455.

Leaf nodes 322, 324, 326 may be reproduced because block 406, whichcorresponds to root node 306, includes an offset to block 412 ofserialized snapshot tree data 461. Block 412 of serialized snapshot treedata 461 corresponds to intermediate node 312. Block 412 includes anoffset to blocks 422, 424, 426, which correspond to leaf nodes 322, 324,326, respectively. The corresponding values associated with leaf nodes322, 324, 326 may be accessed and reproduced because blocks 422, 424,426 include file offsets to one or more blocks of data stored in filesystem data 451.

Leaf node 330 may be reproduced because block 418 of serialized snapshottree data 465 includes an offset to block 430 of serialized snapshottree data 461. Block 430 of serialized snapshot tree data 461corresponds to leaf node 330. The value associated with leaf node 330may be accessed and reproduced because block 430 includes an offset toone or more blocks of data stored in file system data 451.

In some embodiments, a partial tree data structure is reproduced bydeserializing one or more serialized snapshot tree data. For example,for the request of a value associated with a data key of “4” at timet=3, a portion of tree data structure 380 may be reproduced based onserialized snapshot tree data 465. As seen in FIG. 3C, leaf node 334 hasa data key-value pair of “4: DATA4″” and a TreeID of “3.” Because aTreeID of “3” is associated with a snapshot tree view at t=3, the valuestored in leaf node 334, as opposed to the value stored in leaf nodes328, 332, is the value of a data key “4” at t=3. Although serializedsnapshot tree data 465 includes file offsets to serialized snapshot treedata 463, 461, serialized snapshot tree data 461 463 do not need to bedeserialized because the requested value may be determined withoutdeserializing those files. In some embodiments, a subset of theserialized snapshot tree data needed to produce the entire snapshot isdeserialized to determine the value for a data key at the particulartime.

At 708, the reproduced tree data structure is traversed to locate thedata associated with a user request. A user may request to restore anentire snapshot archive to restore the primary system to a particularmoment in time or the user may request to restore a portion of thesnapshot archive to restore one or more files that are included in thesnapshot archive.

The reproduced tree is traversed based on the one or more data keysassociated with the request. For example, for a request for a valueassociated with a data key of “4” at time t=3, reproduced tree datastructure 380 may be traversed from reproduced root node 306 toreproduced intermediate node 318 to reproduced leaf node 334.

At 710, the requested data is retrieved from cloud instantiation of thesecondary storage system and provided. For example, for a request for avalue associated with a data key of “4” at time t=3, a value of “DATA4″”may be retrieved from the file system data stored in the virtual cloudinstances of the cloud instantiation and provided.

In some embodiments, all of the file system data associated with thereproduced snapshot tree is provided. In other embodiments, a portion ofthe file system data associated with the reproduced snapshot tree isprovided.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A method, comprising: generating a cloudinstantiation of a secondary storage system; receiving an indication tovirtually rebuild one or more secondary storage clusters based on asnapshot archive stored in cloud storage, wherein the snapshot archiveis comprised of serialized data, wherein the serialized data iscomprised of a flat set of data blocks, wherein each data block includedin the flat set of data blocks corresponds to one of a plurality ofnodes of a tree data structure, wherein the tree data structure iscomprised of a root node, one or more intermediate nodes, and one ormore leaf nodes; virtually rebuilding the one or more secondary storageclusters including by reconstituting the tree data structure in thecloud instantiation of the secondary storage system based on theserialized data included in the snapshot archive, wherein thereconstituted tree data structure is comprised of at least the rootnode, one of the one or more intermediate nodes, and one of the one ormore leaf nodes; and providing a copy of at least a portion of data ofthe one or more rebuilt storage clusters in the cloud instantiation torestore at least the portion of data to a user destination system. 2.The method of claim 1, further comprising receiving an indication thatthe secondary storage system is offline.
 3. The method of claim 2,wherein the indication that the secondary storage system is offlineincludes a request to generate the cloud instantiation of the secondarystorage system.
 4. The method of claim 3, further comprisingauthenticating a user associated with the request.
 5. The method ofclaim 4, further comprising in response to authenticating the userassociated with the request, providing a list of one or more snapshotarchives available to restore.
 6. The method of claim 1, whereingenerating the cloud instantiation of the secondary storage systemincludes provisioning cloud resources for the cloud instantiation of thesecondary storage system.
 7. The method of claim 6, wherein theprovisioned cloud resources include a plurality of virtual cloudinstances.
 8. The method of claim 6, wherein the provisioned cloudresources include a same storage capacity as the secondary storagesystem.
 9. The method of claim 1, wherein the tree data structure isreconstituted by deserializing the serialized data.
 10. The method ofclaim 1, wherein the snapshot archive corresponds to a state of filesystem data associated with a primary system at a particular moment intime.
 11. A system, comprising: a processor configured to: generate acloud instantiation of a secondary storage system; receive an indicationto virtually rebuild one or more secondary storage clusters based on asnapshot archive stored in cloud storage, wherein the snapshot archiveis comprised of serialized data, wherein the serialized data iscomprised of a flat set of data blocks, wherein each data block includedin the flat set of data blocks corresponds to one of a plurality ofnodes of a tree data structure, wherein the tree data structure iscomprised of a root node, one or more intermediate nodes, and one ormore leaf nodes; and virtually rebuild the one or more secondary storageclusters including by reconstituting the tree data structure in thecloud instantiation of the secondary storage system based on theserialized data included in the snapshot archive; and a communicationinterface coupled to the processor and configured to provide a copy ofat least a portion of data of the one or more rebuilt storage clustersin the cloud instantiation to restore at least the portion of data to auser destination system.
 12. The system of claim 11, wherein thecommunication interface is further configured to receive the indicationthat the secondary storage system is offline.
 13. The system of claim12, wherein the indication that the secondary storage system is offlineincludes a request to generate the cloud instantiation of the secondarystorage system.
 14. The system of claim 13, wherein the processor isfurther configured to authenticate a user associated with the request.15. The system of claim 14, wherein in response to an authentication ofthe user associated with the request, the processor is furtherconfigured to provide a list of one or more snapshot archives availableto restore.
 16. The system of claim 11, wherein to generate the cloudinstantiation of the secondary storage system, the processor is furtherconfigured to provision cloud resources for the cloud instantiation ofthe secondary storage system.
 17. The system of claim 16, wherein theprovisioned cloud resources include a plurality of virtual cloudinstances.
 18. The system of claim 16, wherein the provisioned cloudresources include a same storage capacity as the secondary storagesystem.
 19. A computer program product, the computer program productbeing embodied in a non-transitory computer readable storage medium andcomprising computer instructions for: generating a cloud instantiationof a secondary storage system; receiving an indication to virtuallyrebuild one or more secondary storage clusters based on a snapshotarchive stored in cloud storage, wherein the snapshot archive iscomprised of serialized data, wherein the serialized data is comprisedof a flat set of data blocks, wherein each data block included in theflat set of data blocks corresponds to one of a plurality of nodes of atree data structure, wherein the tree data structure is comprised of aroot node, one or more intermediate nodes, and one or more leaf nodes;virtually rebuilding the one or more secondary storage clustersincluding by reconstituting the tree data structure in the cloudinstantiation of the secondary storage system based on the serializeddata included in the snapshot archive, wherein the reconstituted treedata structure is comprised of at least the root node, one of the one ormore intermediate nodes, and one of the one or more leaf nodes; andproviding a copy of at least a portion of data of the one or morerebuilt storage clusters in the cloud instantiation to restore at leastthe portion of data to a user destination system.
 20. The computerprogram product of claim 19, further comprising computer instructionsfor generating the cloud instantiation of the secondary storage systemincludes provisioning cloud resources for the cloud instantiation of thesecondary storage system.